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FIGHTER  AIRCRAFT 

Better  Cost  Estimates  Needed  for  Extending  the 
Service  Life  of  Selected  F-16s  and  F/A-18s 


Why  GAO  Did  This  Study  What  GAO  Found 


Fighter  aircraft  are  important  to 
achieve  and  maintain  air  dominance 
during  combat  operations  as  well  as  to 
protect  the  homeland.  DOD  plans  to 
replace  many  of  its  current  fighter  fleet 
with  the  F-35;  however,  the  F-35 
program  has  experienced  numerous 
delays  and  cost  increases.  To  maintain 
fighter  capabilities  and  capacity,  the  Air 
Force  and  Navy  have  decided  to 
upgrade  and  extend  the  service  life  of 
selected  F-16  and  F/A-18  aircraft.  In 
this  context,  two  subcommittees  of  the 
Flouse  Armed  Services  Committee 
asked  GAO  to  (1)  describe  the  Air 
Force  and  Navy  plans  to  upgrade  and 
extend  the  service  life  of  selected  F-16 
and  F/A-18  aircraft;  and  (2)  assess  the 
extent  to  which  cost  estimates  for 
these  upgrades  and  life  extensions 
exhibit  characteristics  of  a  high-quality 
cost  estimate.  GAO  obtained 
documentation  of  the  plans  and 
estimates,  compared  the  estimates  to 
best  practices  outlined  in  the  GAO 
Cost  Estimating  Guide,  and  assessed 
factors  that  could  affect  total  costs. 

What  GAO  Recommends 

GAO  recommends  that  the  Air  Force 
and  Navy  follow  all  best  practices  to 
enhance  the  credibility  of  the  cost 
estimates  for  the  F-1 6  and  F/A-1 8 
upgrades  and  life  extensions  including 
an  assessment  of  the  potential  range 
of  costs  and  seeking  independent  cost 
estimates.  DOD  agreed  with  all  four 
recommendations. 


View  GAO-13-51.  For  more  information, 
contact  John  H.  Pendleton  at  404-679-1816  or 
pendletonj@gao.gov. 


The  Air  Force  plans  to  upgrade  and  extend  the  service  life  of  300  F-16  aircraft  and 
the  Navy  150  F/A-18  aircraft,  at  a  combined  cost  estimated  at  almost  $5  billion  in 
fiscal  year  2013  dollars. 

•  The  Air  Force  plans  to  extend  the  service  life  of  selected  F-1 6s  by  2,000  flying 
hours  each  as  well  as  install  capability  upgrades  such  as  an  improved  radar. 
The  Air  Force  estimates  that  it  will  complete  this  work  by  2022  at  a  cost  of 
$2.61  billion.  About  28  percent  of  the  projected  costs  are  included  in  the  Air 
Force’s  spending  plans  through  2017,  with  the  remainder  expected  to  be 
incurred  in  2018-2022. 

•  The  Navy  plans  to  extend  the  service  life  of  selected  F/A-1 8s  by  1 ,400  flying 
hours  each  and  may  install  capability  upgrades  on  some  of  the  150  aircraft — 
such  as  adding  the  ability  to  integrate  with  newer  aircraft.  The  Navy  projects 
that  it  will  complete  the  life  extension  by  2018  at  a  cost  of  $2.19  billion,  with 
most  of  these  costs  included  in  its  spending  plans  through  2017,  but  costs 
associated  with  any  upgrades  are  not  included  in  the  Navy  estimate  or  in  its 
spending  plans. 

Air  Force  and  Navy  officials  told  GAO  that  they  could  ultimately  extend  the  service  life 
of  up  to  650  F-1 6s  and  280  F/A-1 8s  if  needed  to  attain  desired  inventory  levels. 


The  Air  Force’s  and  Navy’s  cost  estimates  to  upgrade  and  extend  the  service  life  of 
selected  fighter  aircraft  exhibit  some  characteristics  of  a  high-quality  cost  estimate  but 
do  not  reflect  all  potential  costs.  The  estimates  were:  well-documented  since  they 
identified  data  sources  and  methodologies;  accurate  since  they  accounted  for 
inflation  and  were  checked  for  errors;  and  mostly  comprehensive  since  they  included 
the  work  planned  and  identified  key  assumptions.  However,  the  estimates  were  not 
fully  credible  in  part  because  they  did  not  assess  the  extent  to  which  the  total  costs 
could  change  if  additional  work  is  done  or  more  aircraft  are  included  in  the  programs. 
For  example,  Air  Force  leaders  indicated  in  March  2012  that  they  intend  to  upgrade 
and  extend  the  service  life  of  50  additional  F-1 6s  beyond  the  original  300,  but  the  Air 
Force  has  not  assessed  how  much  the  cost  might  increase  if  more  aircraft  are  added 
to  the  program.  In  addition,  the  Navy  plans  to  upgrade  the  capabilities  of  some 
aircraft  at  the  same  time  as  the  service-life  extension,  but  this  cost  is  not  included  in 
the  Navy  estimates.  Also,  the  Navy  may  extend  the  life  of  or  replace  other  aircraft 
components  that  are  becoming  obsolete,  but  these  costs — which  could  add  an 
average  cost  of  $5.64  million  per  aircraft — were  also  not  included  in  the  original  $2.19 
billion  estimate.  Another  factor  affecting  the  credibility  of  the  estimates  is  that  they 
have  not  been  compared  to  an  independently  developed  estimate.  GAO’s  past  work 
has  shown  that  such  an  independent  cost  estimate  is  one  of  the  best  validation 
methods  since  an  independent  cost  estimate  tends  to  be  higher  and  more  accurate 
than  a  program  office  estimate.  Air  Force  and  Navy  officials  told  GAO  that  they  use 
Department  of  Defense  and  military  department  guidance  that  allows  for  some 
variation  in  how  the  estimates  are  developed  depending  on  the  dollar  value  and 
maturity  of  the  program.  However,  these  programs-which  are  critical  to  maintain 
fighter  capability  and  capacity  as  current  inventory  ages--total  almost  $5  billion  and 
the  costs  will  increase  if  program  quantities  and  scope  increase.  Without  fully  credible 
cost  estimates,  including  an  analysis  of  how  much  total  costs  may  increase,  decision 
makers  will  not  have  visibility  into  the  range  of  potential  costs,  which  could  hinder 
their  ability  to  formulate  realistic  budgets  and  make  informed  investment  decisions. 
_ United  States  Government  Accountability  Office 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


November  15,  2012 

The  Honorable  J.  Randy  Forbes 
Chairman 

The  Honorable  Madeleine  Z.  Bordallo 
Ranking  Member 
Subcommittee  on  Readiness 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Roscoe  G.  Bartlett 
Chairman 

The  Honorable  Silvestre  Reyes 
Ranking  Member 

Subcommittee  on  Tactical  Air  and  Land  Forces 
Committee  on  Armed  Services 
House  of  Representatives 

Fighter  aircraft  are  important  to  achieving  and  maintaining  air  dominance 
during  combat  operations  as  well  as  protecting  the  homeland.  These 
aircraft  operate  during  the  first  days  of  a  conflict  to  penetrate  enemy  air 
space  and  defeat  air  defenses,  which  allows  follow-on  forces  freedom  to 
operate.  Overall,  the  Air  Force  and  Navy  have  about  3,500  fighter  aircraft 
in  inventory  as  of  fiscal  year  201 2,  including  about  1 ,020  Air  Force  F-1 6s 
and  about  624  Navy  F/A-18  A-Ds.1  Many  of  the  Department  of  Defense’s 
(DOD)  current  fighter  aircraft  are  more  than  20  years  old  on  average,  and 
although  DOD  plans  to  replace  much  of  the  existing  inventory  by 
procuring  2,443  new  F-35s,  the  department  has  experienced  numerous 
delays  and  cost  increases  in  the  F-35  program.  Therefore,  the  Air  Force 
and  the  Navy  have  considered  several  alternatives  to  maintain  desired 
fighter  inventory  levels  at  an  acceptable  level  of  risk  and  to  maintain 
fighter  capabilities  and  capacity  as  a  “bridge”  until  replacement  F-35 
aircraft  enter  service  in  sufficient  numbers.  The  Air  Force  has  decided  to 
upgrade  and  extend  the  service  life  of  selected  F-1 6s  and  the  Navy  has 


^he  Navy  figure  includes  aircraft  that  are  used  by  the  United  States  Navy  and  Marine 
Corps. 
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decided  to  extend  the  service  life  of  selected  F/A-18s.2  These  programs 
are  estimated  to  cost  several  billion  dollars  at  a  time  when  DOD  is 
simultaneously  facing  the  competing  demands  of  developing  and 
procuring  F-35s  at  a  cost  approaching  $400  billion,  supporting  ongoing 
operations,  and  implementing  $487  billion  of  reductions  over  the  next  10 
years,  which  the  department  has  estimated  that  it  needs  to  cut  in  order  to 
comply  with  the  Budget  Control  Act  of  201 1 ,3 

We  previously  have  reported  that  the  F-35  program  has  experienced 
numerous  delays  in  development  and  production,  including  program 
restructuring,  which  has  added  time  for  development.4  As  a  result,  DOD 
has  deferred  procurement  of  410  F-35s  until  after  fiscal  year  201 7. 5 
Considering  that  further  slips  in  the  F-35  program  may  occur  and  could 
increase  the  scope  and  cost  of  the  services’  plans  for  upgrading  and 
extending  the  service  life  of  current  aircraft,  you  asked  us  to  (1)  describe 
the  Air  Force  and  Navy  plans  to  upgrade  and  extend  the  service  life  of 
selected  F-16  and  F/A-18  aircraft,  and  (2)  assess  the  extent  to  which  cost 
estimates  for  these  programs  exhibit  characteristics  of  a  high-quality  cost 
estimate.  This  report  reviewed  estimates  of  costs  projected  for  fiscal 
years  2011  through  2022. 

To  describe  the  Air  Force’s  and  the  Navy’s  plans  to  upgrade  and  extend 
the  service  life  of  selected  F-16s  and  F/A-18s,  we  reviewed  the  services’ 
documentation  of  the  programs’  purpose  and  scope  including  the 
underlying  analysis  of  alternative  approaches.  We  also  obtained  cost 
estimates  for  these  programs  and  obtained  any  updates  to  these 
estimates.  Since  the  service  cost  estimates  were  developed  in  different 


2The  Navy  also  decided  to  buy  an  additional  41  F/A-18  E/F  aircraft.  This  report  focuses 
on  the  services’  plans  to  upgrade  and  extend  the  service  life  of  current  aircraft  and  does 
not  include  an  evaluation  of  buying  new  aircraft. 

department  of  Defense,  “Defense  Budget  Priorities  and  Choices”  (January  2012). 

4GAO,  Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance  Restructuring  and 
Address  Affordability  Risks,  GAO-12-437  (Washington,  D.C.:  June  14,  2012). 

5GAO,  Joint  Strike  Fighter:  Restructuring  Added  Resources  and  Reduced  Risk,  but 
Concurrency  Is  Still  a  Major  Concern,  GAO-12-525T  (Washington,  D.C.:  Mar.  20,  2012) 
and  Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance  Restructuring  and 
Address  Affordability  Risks,  GAO-12-437  (Washington,  D.C.:  June  14,  2012).  The  June 
2012  report  states  that  DOD  has  purchased  63  F-35  (Joint  Strike  Fighter)  aircraft  as  of 
January  2012,  and,  with  the  latest  reduction,  DOD  now  plans  to  buy  a  total  of  365  aircraft 
through  2017,  about  one-fourth  of  the  1 ,591  aircraft  expected  in  the  2002  plan. 
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base  years,  all  of  the  cost  estimates  in  this  report  have  been  converted  to 
fiscal  year  2013  dollars  using  the  indexes  published  by  the  Office  of  the 
Under  Secretary  of  Defense  (Comptroller)  so  that  the  data  can  be 
presented  in  comparable  terms.6  To  assess  the  extent  to  which  cost 
estimates  exhibit  characteristics  of  a  high-quality  cost  estimate,  we 
compared  how  the  estimates  were  developed  to  the  cost-estimating  best 
practices  outlined  in  the  GAO  Cost  Estimating  and  Assessment  Guide 7 
We  analyzed  the  extent  to  which  the  Air  Force  and  Navy  cost  estimates 
were  comprehensive,  well-documented,  accurate,  and  credible — the  four 
characteristics  of  a  high-quality  cost  estimate — and  assigned  each 
characteristic  a  rating  of  not  met,  minimally  met,  partially  met, 
substantially  met,  or  met.8  We  also  held  detailed  discussions  with  service 
officials  and  reviewed  program  documentation  to  identify  key  factors  that 
could  affect  the  potential  total  costs  such  as  changes  in  aircraft  quantities 
or  additional  work  to  maintain  aircraft  mission  effectiveness  that  may  not 
have  been  included  in  the  estimates.  We  shared  our  cost  guide,  the 
criteria  against  which  we  evaluated  the  program’s  cost  estimates,  and  our 
preliminary  findings  with  program  officials.  When  warranted,  we  updated 
our  analyses  based  on  the  agency  response  and  additional 
documentation  provided  to  us.  Finally,  we  corroborated  our  analyses  in 
interviews  with  service  officials  responsible  for  developing  the  cost 
estimates.  See  appendix  I  for  a  complete  description  of  our  scope  and 
methodology. 

We  conducted  this  performance  audit  from  May  2011  to  November  201 2 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 


6App.  II  shows  the  Air  Force  cost  estimates  for  upgrading  and  extending  the  service  life  of 
selected  F-16  aircraft  in  then-year  dollars.  The  Navy  estimate  was  reported  as  a  cost  per 
aircraft  in  fiscal  year  201 1  dollars.  Therefore,  there  is  not  comparable  data  for  showing 
the  Navy  estimate  in  then-year  dollars. 

7GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs ,  GAO-09-3SP  (Washington,  D.C:  March  2009). 

8Not  met  means  the  military  service  provided  no  evidence  that  satisfies  any  of  the 
criterion;  minimally  met  means  the  military  service  provided  evidence  that  satisfies  a  small 
portion  of  the  criterion;  partially  met  means  the  military  service  provided  evidence  that 
satisfies  about  half  of  the  criterion;  substantially  met  means  the  military  service  provided 
evidence  that  satisfies  a  large  portion  of  the  criterion;  and  met  means  the  military  service 
provided  complete  evidence  that  satisfies  the  entire  criterion. 
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findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 

Background 

The  Air  Force  and  the  Navy  plan  to  replace  many  of  their  current  fighter 
aircraft  with  the  Joint  Strike  Fighter,  also  called  the  F-35.  DOD  plans  to 
buy  a  total  of  2,443  F-35s — 1 ,763  for  the  Air  Force  and  680  for  the 
Department  of  the  Navy — between  2008  and  2037.  However,  we  have 
reported  on  F-35  issues  since  2005  and  found  that  the  F-35  is  still  in 
development  and  the  program  has  experienced  significant  delays  and 
cost  increases.  For  example,  we  reported  that  the  F-35  program  has 
experienced  numerous  delays  in  development  and  production,  including 
program  restructuring,  which  has  increased  time  for  development. 

Further,  we  reported  that  the  F-35  program  costs  have  increased  42 
percent  from  the  2007  baseline  and  unit  costs  have  doubled  since  the 
start  of  development  in  2001. 9  In  addition,  the  DOD  Fiscal  Year  2011 
Annual  Report  on  Operational  Test  and  Evaluation  stated  that  the  F-35  is 
not  on  track  to  meet  operational  effectiveness  or  operational  suitability 
requirements  and  that  testing  identified  structural  and  maintenance 
issues.10  Further,  the  DOD  report  also  stated  that  aircraft  produced  in  the 
first  few  production  lots  will  need  a  significant  number  of  modifications 
and  upgrades  in  order  to  attain  planned  service  life  and  capabilities.  Due 
to  delays  in  the  F-35  program,  the  services  have  decided  to  extend  the 
service  life  of  some  existing  aircraft  in  order  to  maintain  desired  inventory 
levels. 

The  upgrade  and  service-life  extension  programs  will  likely  be  costly  and 
may  be  subject  to  further  changes.  Our  prior  work  has  found  that  a  high- 
quality  cost  estimate  is  critical  to  the  success  of  any  program  because  it 
can  provide  the  basis  for  informed  investment  decision  making,  realistic 
budget  formulation,  and  proactive  course  correction  when  warranted.  The 
March  2009  GAO  Cost  Estimating  and  Assessment  Guide  is  a 
compilation  of  best  practices  that  federal  cost-estimating  organizations 
and  industry  use  to  develop  and  maintain  reliable  cost  estimates 

9GAO,  Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance  Restructuring  and 
Address  Affordability  Risks,  GAO-12-437  (Washington,  D.C.:  June  14,  2012). 

10Department  of  Defense,  Fiscal  Year  201 1  Annual  Report,  Director,  Operational  Test  and 
Evaluation  (December  201 1 ). 
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throughout  the  life  of  a  program.  The  GAO  cost  guide  reported  that  a 

high-quality  cost  estimate  has  four  characteristics  including 

•  comprehensive  when  it  accounts  for  all  life-cycle  costs,  is  sufficiently 
detailed  to  ensure  that  costs  are  neither  omitted  nor  double-counted, 
and  identifies  ground  rules  and  assumptions; 

•  well-documented  when  supporting  documentation  explains  the 
process,  sources,  and  methods  used  to  create  the  estimate; 

•  accurate  when  it  is  based  on  the  assessment  of  the  costs  most  likely 
to  be  incurred  and  adjusted  for  inflation;  and 

•  credible  when  the  level  of  confidence  has  been  identified  through  a 
risk  and  uncertainty  analysis,  and  a  sensitivity  analysis  has  been 
conducted  that  identified  the  effects  on  the  estimate  of  changing  key 
assumptions.  Also,  a  cost  estimate  is  credible  when  it  has  been  cross¬ 
checked  with  an  independent  estimate.  The  GAO  cost  guide 
describes  eight  types  of  independent  cost  estimate  reviews  which 
vary  in  the  depth  of  analysis,  ranging  from  a  document  review — 
merely  assessing  the  estimate’s  documentation — to  an  independently 
developed  cost  estimate — conducted  by  an  organization  outside  the 
acquisition  or  program  office.  The  Air  Force  and  Navy  each  have 
service  cost-estimating  agencies,  which  develop  independent  cost 
estimates  for  major  acquisition  programs  and  other  programs  when 
requested. 


Air  Force  and  Navy 
Programs  Would 
Extend  the  Service 
Life  of  450  Aircraft  at 
a  Cost  of  Nearly  $5 
billion 


The  Air  Force  currently  plans  to  upgrade  and  extend  the  service  life  of 
300  F-16  aircraft  at  an  estimated  cost  of  $2.61  billion  and  the  Navy  plans 
to  extend  the  service  life  of  150  F/A-18  aircraft  at  an  estimated  cost  of 
$2.19  billion.11  Service  officials  have  said  that  if  the  F-35  program 
experiences  further  delays,  the  services  have  the  ability  to  expand  the 
number  of  aircraft  that  are  included  in  these  programs. 


1 1  All  the  cost  figures  in  the  body  of  this  report  are  in  fiscal  year  2013  dollars. 


Page  5 


GAO-13-51  Fighter  Aircraft 


Air  Force  Program  Would 
Add  2,000  Hours  to  300  F- 
16s 


In  2009,  DOD  and  the  Air  Force  were  directed  by  statute  and  a  committee 
report  to  provide  reports  on  force-structure  plans  including  alternatives 
such  as  buying  aircraft  and  upgrading  and  extending  the  service  life  of 
selected  aircraft.12  The  Air  Force  reported  on  assessments  of  seven 
alternatives,  which  included  buying  between  150  and  300  new  F-15Es, 
F/A-18  E/Fs,  and  F-16s,  and  upgrading  and  extending  the  service  life  of 
selected  F-16s.  The  Air  Force  concluded  that  the  cost  for  upgrading  and 
extending  the  service  life  of  current  F-16s  by  2,000  hours  each  would  be 
10  to  15  percent  of  the  cost  of  procuring  new  F-16s,  F-15Es,  or  F/A-18s. 
Extending  the  life  of  existing  aircraft  would  provide  6  to  8  years  of 
additional  service  and,  according  to  the  Air  Force,  provide  essentially  the 
same  capability  over  that  period  as  buying  new  legacy  aircraft.  In 
reviewing  these  reports,  GAO  concluded  in  2011  that  the  analyses  done 
were  limited  in  part  by  the  absence  of  F-16  durability  and  viability  data.13 
Figure  1  below  shows  the  Air  Force’s  estimated  costs  for  the  current 
plans  for  capability  upgrades  and  the  service-life  extension. 


12See  National  Defense  Authorization  Act  for  Fiscal  Year  2010,  Pub.  L.  No.  111-84,  §§ 
131,  1075  (2009);  H.R.  Rep.  No.  111-166,  at  101  (2009).  One  provision  in  the  National 
Defense  Authorization  Act  for  Fiscal  Year  2010  required  the  Secretary  of  Defense  to 
submit  a  report  on  4.5  generation  fighters,  while  another  prohibited  retirement  of  certain 
aircraft  until  the  Secretary  of  the  Air  Force  submitted  a  report  regarding  combat  air  forces 
restructuring.  See  Pub.  L.  No.  111-84,  §§  131,  1075.  In  response,  the  Air  Force  provided 
the  following  reports  as  well  as  a  third,  which  we  did  not  review  as  part  of  our  current 
report:  Department  of  the  Air  Force,  Fighter  Force  Structure  Shortfalls  (April  201 0),  and 
Procurement  of  “4.5  Generation  Fighter”  Aircraft  (April  2010).  These  reports  are 
classified. 

13GAO,  Tactical  Aircraft:  Air  Force  Fighter  Reports  Generally  Addressed  Congressional 
Mandates,  but  Reflected  Dated  Plans  and  Guidance,  and  Limited  Analyses, 

GAO-1 1-323R  (Washington,  D.C.:  Feb.  24,  2011). 
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Figure  1:  Air  Force  F-16  and  Estimated  Costs  to  Upgrade  and  Extend  the  Service  Life  for  2,000  Hours  (Fiscal  Year  2013 
Dollars  in  Billions) 


Source:  GAO  analysis  of  Air  Force  data.  Air  Force  (photo).  GAO  (artwork) 

a$820  million  service-life  extension;  $1.79  billion  capability  upgrade. 


The  Air  Force  estimates  to  upgrade  capabilities  and  extend  the  service 
life  of  selected  F-16  aircraft  included  research,  development,  test  and 
evaluation,  and  procurement  costs.  The  capability  upgrades,  Air  Force 
officials  explained,  are  needed  to  maintain  mission  effectiveness  and  will 
include,  for  example,  an  improved  radar  and  data-link  enhancements. 

The  service-life  extension  will  add  2,000  flight  hours,  or  about  6  to  8 
years,  depending  on  flying  conditions,  to  each  aircraft.  Most  of  the  $2.61 
billion  to  upgrade  and  extend  the  service  life — $1.88  billion  or  72 
percent — is  planned  to  be  incurred  in  fiscal  years  2018  through  2022.  The 
remaining  $722  million  is  programmed  in  the  fiscal  year  2013-2017  Future 
Years  Defense  Program.14 

Since  the  2010  reports  to  Congress  were  completed,  the  Air  Force  has 
stated  that  without  the  service-life  extension  it  would  have  to  start 
gradually  grounding  some  F-16s  beginning  in  2017.  The  Air  Force  is 
conducting  tests  to  clarify  the  extent  of  the  work  required  to  extend  the 


14The  Future  Years  Defense  Program  provides  information  on  DOD’s  current  and  planned 
budget  requests  for  a  5-year  period. 
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service  life,  and  plans  to  begin  work  in  fiscal  year  2016.  Also,  the  Air 
Force  plans  a  milestone  program  review  for  the  service-life  extension  in 
the  first  quarter  of  fiscal  year  2013  and  a  milestone  program  review  for 
the  capability-upgrade  program  in  the  fourth  quarter  of  fiscal  year  2013. 
The  Air  Force  plans  to  complete  all  the  upgrade  and  service-life  extension 
work  by  the  early  2020s.  Finally,  Air  Force  officials  said  that  they  could 
expand  these  programs  to  include  up  to  650  aircraft  if  needed  to  attain 
desired  inventory  levels. 


Navy  Program  Would  Add  In  response  to  a  statutory  requirement,  in  May  201 1 ,  the  Navy  submitted 
1  400  Hours  to  150  F/A-18S  the  results  of  a  cost-benefit  analysis  comparing  extending  the  service  life 

of  existing  F/A-18  aircraft  with  procuring  additional  F/A-18  E/F  aircraft.15 
The  Navy  assessed  six  alternatives,  which  included  various  combinations 
of  extending  the  service  life  of  up  to  280  F/A-18  A-D  aircraft  and  procuring 
up  to  an  additional  70  F/A-18  E/Fs.  The  Navy  concluded  that  extending 
the  service  life  of  150  F/A-18  A-D  aircraft  and  buying  41  new  F/A-18E/F 
aircraft  would  provide  an  acceptable  inventory  at  a  manageable  level  of 
risk.16  The  Navy  plans  to  complete  the  service-life  extension  work  in  fiscal 
year  2018.  Figure  2  below  shows  the  Navy’s  estimated  cost  for  the 
service-life  extension  of  selected  F/A-18  aircraft. 


15Department  of  Navy,  “Report  to  Congress  on  Service  Life  Extension  of  F/A-1 8  Aircraft" 
(May  13,  2011).  This  report  was  developed  in  response  to  a  provision  in  the  Ike  Skelton 
National  Defense  Authorization  Act  for  Fiscal  Year  201 1  that  required  the  Secretary  of  the 
Navy  to  conduct  the  cost-benefit  analysis  and  submit  a  report  prior  to  entering  into  a 
program  to  extend  the  service  life  of  F/A-18  aircraft  beyond  8,600  hours.  See  Pub.  L.  No. 
111-383,  §  114(a)  (2011).  The  provision  required  the  Navy  to  conduct  the  analysis  in 
accordance  with  Office  of  Management  and  Budget  Circular  A-94.  See  §  114(a)(1).  The 
report  included  comprehensive  costs  for  each  of  the  alternatives  assessed. 

16The  Navy  report  stated  that  the  41  additional  aircraft  were  included  in  the  fiscal  year 
2012  President’s  Budget  request. 
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Figure  2:  Navy  F/A-18  A-D  and  Estimated  Costs  to  Extend  the  Service  Life  for  1,400  hours  (Fiscal  Year  2013  Dollars  in 
Billions) 


Source:  GAO  analysis  of  Navy  data  Navy  (photo);  GAO  (artwork) 


The  Navy’s  estimate  includes  procurement  costs  for  materials  and 
installation  to  extend  the  service  life  of  150  F/A-18s  for  an  additional 
1,400  hours  each  or  about  5  years  depending  on  flying  conditions.17 
Almost  all  the  Navy’s  estimated  $2.19  billion  is  programmed  in  the  fiscal 
year  2013-2017  Future  Years  Defense  Program.  Finally,  the  Navy’s 
analysis  showed  that  the  150  aircraft  selected  for  the  service-life 
extension  will  individually  be  evaluated  for  capability  upgrades — such  as 
adding  the  ability  to  integrate  with  newer  aircraft — as  well  as  upgrades  of 
other  critical  components  reaching  the  end  of  their  service  life  and 
becoming  obsolete.  After  evaluating  the  aircraft,  the  Navy  will  decide 
whether  to  do  this  additional  work.  Finally,  Navy  officials  said  that  they 
could  expand  their  program  if  needed,  to  include  up  to  280  aircraft.  Navy 
officials  explained  that  they  will  review  the  service-life  extension  effort 
annually  as  part  of  an  overall  review  of  the  F/A-1 8  program. 


17According  to  Navy  officials,  a  small  amount  of  the  funding  is  operations  and  support 
money  which  will  directly  support  the  service-life  extension  work  and  is  not  to  support 
aircraft  operations.  In  addition  to  the  procurement  costs,  the  Navy  estimated  that 
operating  and  support  costs  for  the  added  1 ,400  hours  of  service  life  would  be  $5.91 
billion  which  is  not  a  part  of  the  estimate  for  procurement  and  installation  costs  to  extend 
the  service  life. 


Page  9 


GAO-13-51  Fighter  Aircraft 


Air  Force  and  Navy 
Cost  Estimates 
Exhibit  Many 
Characteristics  of  a 
High-Quality  Cost 
Estimate  but  Do  Not 
Reflect  Some 
Significant  Potential 
Costs 


The  Air  Force’s  and  Navy’s  cost  estimates  to  upgrade  and  extend  the 
service  life  of  selected  F-16  and  F/A-18  aircraft  exhibit  many  cost¬ 
estimating  characteristics  and  best  practices  of  a  high-quality  cost 
estimate,  but  do  not  reflect  all  the  potential  total  costs  that  may  occur.  We 
assessed  the  Air  Force  estimate  for  capability  upgrades  and  the  Air  Force 
estimate  for  the  service-life  extension  of  selected  F-16  aircraft,  which 
were  prepared  for  the  fiscal  year  2013  budget  request;  and  the  Navy 
estimate  for  the  service-life  extension  of  selected  F/A-18  aircraft.  Overall, 
we  found  that  the  Air  Force  and  Navy  followed  many  of  the  best  practices 
that  support  the  four  characteristics  of  a  high-quality  estimate.  However, 
both  the  Air  Force  and  Navy  estimates  were  not  fully  credible  due  to  two 
shortcomings — the  estimates  did  not  show  the  range  of  potential  costs 
that  are  likely  to  be  incurred  and  were  not  validated  by  comparison  with 
an  independently  developed  estimate.  As  a  result,  decision-makers  do 
not  have  visibility  of  how  much  the  total  costs  will  be  and  how  they  may 
increase  if  program  quantities  increase  or  additional  work  is  required  on 
some  aircraft,  which  could  hinder  their  ability  to  assess  budgets  and 
affordability. 


Air  Force  Estimates 
Exhibit  Some 
Characteristics  of  a  High- 
Quality  Estimate  but  Are 
Not  Fully  Credible 


The  Air  Force’s  $1 .79  billion  estimate  for  capability  upgrades  and  the 
$820  million  estimate  for  service-life  extension  were  well-documented, 
accurate,  and  mostly  comprehensive.  However,  the  estimates  were  not 
fully  credible  because  the  Air  Force’s  analysis  did  not  clearly  show  the 
potential  range  of  total  costs  that  would  occur  if  more  aircraft  are  included 
in  the  programs  and  the  estimates  were  not  compared  to  an 
independently  developed  estimate.  In  assessing  the  extent  to  which  the 
Air  Force’s  cost  estimates  exhibited  the  four  characteristics  of  a  high- 
quality  estimate,  we  found  the  following: 


•  Comprehensive :  The  Air  Force’s  estimates  were  substantially 
comprehensive  because  they  included  all  the  work  planned  for  300 
aircraft,  identified  key  cost-estimating  ground  rules  and  assumptions, 
and  included  development  and  procurement  costs. 

•  Well-documented :  Both  estimates  were  well-documented  because 
they  identified  the  source  of  the  data,  included  the  work  that  is 
planned,  documented  the  cost-estimating  methodologies,  showed 
how  the  calculations  were  made,  and  were  reviewed  by  management. 

•  Accurate :  Both  estimates  were  accurate  because  they  were  adjusted 
for  inflation,  based  on  an  assessment  of  most  likely  costs  for  300 
aircraft,  and  used  data  from  comparable  programs  where  appropriate. 

•  Credible:  For  both  estimates,  the  Air  Force  cross-checked  major  cost 
elements  to  determine  whether  results  were  similar  and  conducted  a 
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risk  and  uncertainty  analysis  that  included  a  list  of  the  risks  assessed. 
For  example,  the  analysis  assessed  the  risk  that  more  work  may  be 
required  to  extend  the  service  life  than  currently  planned.  Also,  the  Air 
Force  identified  some  cost  drivers,  which  are  the  items  that  have  the 
greatest  effect  on  the  estimated  cost,  such  as  labor  rates.  While  the 
Air  Force  estimates  exhibited  many  characteristics  of  a  high-quality 
estimate,  they  had  two  shortcomings  that  lessened  the  estimates’ 
credibility.  First,  the  analysis  did  not  include  an  estimate  of  the  total 
costs  that  may  occur  if  more  than  300  aircraft  are  included  in  the 
programs  up  to  the  potential  maximum  of  650  aircraft.  Second,  the 
estimates  were  not  compared  to  an  independently  developed 
estimate. 

Regarding  the  first  shortcoming,  best  practices  state  that  a  credible  cost 
estimate  should  include  an  assessment  of  how  the  cost  estimate  may 
change  in  response  to  changes  in  key  program  assumptions.  This  is 
known  as  a  sensitivity  analysis.  Such  an  analysis  helps  decision  makers 
identify  areas  that  could  significantly  affect  a  program’s  cost,  such  as 
changes  in  program  quantities.  According  to  Air  Force  officials,  the 
service  may  choose  to  upgrade  and  extend  the  service  life  of  additional 
aircraft  if  there  are  further  delays  in  F-35  production.  For  example,  in  a 
written  statement,  Air  Force  leaders  testified  in  March  2012  that  the  intent 
is  to  include  350  F-16  aircraft  in  the  upgrade  and  life  extension 
programs18 — 50  more  aircraft  than  originally  planned.  Air  Force  officials 
explained  that  they  did  not  assess  the  range  of  costs  that  may  result  from 
expanding  the  number  of  aircraft  in  the  program  up  to  the  maximum  of 
650  since  the  program  is  currently  approved  for  only  300  aircraft. 
However,  without  an  assessment  of  how  much  the  total  cost  may 
increase  as  the  aircraft  quantity  increases,  the  estimate  lacks 
transparency  on  the  full  range  of  possible  costs,  which  could  be 
significant.  For  example,  we  calculated  that  the  cost  to  upgrade  and 
extend  the  service  life  of  all  650  aircraft  could  total  $5.53  billion.19 


1  ft 

Prepared  statement  of  Major  General  James  M.  Holmes,  Assistant  Deputy  Chief  of  Staff 
for  Operations,  Plans  and  Requirements  and  Major  General  John  D.  Posner,  Director  of 
Global  Power  Programs,  Office  of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition) 
before  the  Subcommittee  on  Tactical  Air  and  Land  Forces,  House  Armed  Services 
Committee  on  March  20,  2012. 

1 Q 

This  number  is  a  GAO  calculation  based  on  Air  Force  data  using  the  per  unit  cost  that 
the  Air  Force  estimated  for  300  aircraft.  The  per  unit  cost  may  change  as  quantities 
increase  due,  for  example,  to  economies  of  scale. 
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The  second  major  shortcoming  is  that  the  Air  Force  did  not  compare  the 
estimates  to  independently  developed  cost  estimates.  Having  an 
independent  estimate  is  considered  one  of  the  best  and  most  reliable 
estimate  validation  methods  since  an  independent  estimate  is  an 
objective  assessment  of  whether  the  program  office  estimate  is 
reasonable  and  can  be  achieved.  Further,  our  past  work  has  shown  that 
an  independent  estimate  tends  to  be  higher  and  more  accurate  than  a 
program  office  estimate.20  Air  Force  officials  explained  that  it  is  likely  the 
Air  Force  Cost  Analysis  Agency  will  be  requested  to  develop  a  formal 
independent  cost  estimate  for  upcoming  program  reviews,  but  the 
decision  to  require  an  independent  estimate  has  not  yet  been  made. 
Further,  the  program  review  for  the  capability-upgrade  program  will  not  be 
held  for  a  year — it  is  currently  scheduled  for  the  fourth  quarter  of  fiscal 
year  2013.  Until  the  Air  Force  obtains  an  independent  cost  estimate, 
decision  makers  will  not  have  the  assurance  that  the  programs’  estimated 
cost  can  be  relied  upon  for  making  budgeting  and  trade-off  decisions. 
Furthermore,  a  program  that  has  not  been  reconciled  with  an  independent 
estimate  has  an  increased  risk  of  proceeding  underfunded  because  an 
independent  estimate  provides  an  objective  and  unbiased  assessment  of 
whether  the  program  estimate  can  be  achieved. 


Navy  Estimate  Exhibits 
Characteristics  of  a  High- 
Quality  Estimate  but  Does 
Not  Capture  Costs  of 
Additional,  Nonstructural 
Work 


The  Navy’s  $2.19  billion  cost  estimate  for  the  service-life  extension  of 
F/A-18s  was  comprehensive,  well-documented,  and  accurate.  However, 
the  estimate  was  not  fully  credible  because  it  did  not  include  an 
assessment  of  the  costs  for  additional  work  on  the  aircraft  that  may  be 
done  at  the  same  time  as  the  service-life  extension  and  the  estimate  was 
not  compared  to  an  independently  developed  estimate.  Since  the  Navy 
will  assess  on  an  aircraft-by-aircraft  basis  whether  to  do  this  additional 
work,  the  costs  would  be  in  addition  to  the  $2.19  billion  for  the  service-life 
extension  and  most  of  these  costs  are  not  included  in  the  Navy’s 
spending  plans  through  fiscal  year  2017.  In  assessing  the  extent  to  which 
the  Navy’s  cost  estimate  exhibited  the  four  characteristics  of  a  high- 
quality  estimate,  we  found  the  following: 


20ln  a  previous  review  of  DOD  acquisition  programs,  we  found  that  19  of  20  independent 
estimates  developed  by  the  Office  of  the  Secretary  of  Defense  office  of  the  Director  of 
Cost  Assessment  and  Program  Evaluation  were  higher  than  the  service  estimate.  See 
GAO,  Defense  Acquisitions:  A  Knowledge-Based  Funding  Approach  Could  Improve  Major 
Weapon  System  Program  Outcomes ,  GAO-08-619  (Washington,  D.C.:  July  2,  2008). 
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•  Comprehensive :  The  Navy’s  estimate  identified  key  cost-estimating 
ground  rules  and  assumptions.  Also,  the  Navy’s  estimate  included  the 
costs  to  buy  and  install  the  materials  needed  to  repair  the  airframe  so 
that  it  can  fly  the  additional  1,400  hours.  Finally,  the  Navy’s  analysis 
included  assessing  the  costs  for  different  quantities  from  150  up  to 
280  aircraft. 

•  Well-documented :  The  estimate  was  well-documented  since  it 
described  the  methodology  used  and  the  calculations  performed  to 
develop  the  estimate  so  that  a  cost  analyst  unfamiliar  with  the 
program  could  replicate  it.  Also,  the  Navy’s  documentation  included 
steps  to  ensure  data  reliability,  and  the  estimate  was  reviewed  by 
management. 

•  Accurate :  The  Navy’s  estimate  was  accurate  since  it  was  adjusted  for 
inflation,  not  overly  conservative  or  optimistic,  and  incorporated  data 
from  comparable  programs  where  appropriate. 

•  Credible :  Finally,  the  Navy’s  estimate  was  partially  credible  because 
the  Navy  followed  some,  but  not  all,  of  the  best  practices  for  this 
characteristic.  For  example,  the  Navy  conducted  a  sensitivity  analysis 
which  identified  cost  drivers,  such  as  materiel  and  labor  costs.  Also, 
the  Navy  conducted  a  risk  and  uncertainty  analysis  which  accounted 
for  some  risks,  such  as  the  risk  that  the  service-life  extension  may 
require  more  work  than  currently  planned.  However,  the  Navy  did  not 
assess  the  potential  range  of  costs  for  all  work  that  might  be  done  and 
focused  only  on  the  costs  associated  with  extending  the  life  of  the 
airframe,  which  officials  described  as  consistent  with  Navy  guidance. 

Specifically,  the  Navy  told  us  it  will  assess  on  an  aircraft-by-aircraft  basis 
whether  the  F/A-18s  also  need  capability  upgrades  to  maintain  mission 
effectiveness,  such  as  adding  the  ability  to  integrate  with  newer  aircraft.21 
According  to  the  Navy,  such  capability  upgrades  could  cost  an  average  of 
about  $1 .76  million  per  aircraft,  but  uncertainty  exists  about  how  many 
aircraft  the  Navy  will  actually  decide  to  upgrade  and  therefore  the  total 
cost  associated  with  these  upgrades  is  uncertain.  In  addition,  the  Navy 
has  determined  that  some  of  the  F/A-18  aircraft  may  also  require  life 
extension  or  replacement  of  nonstructural  parts  or  systems  that  are 
becoming  obsolete.  The  Navy  estimated  that  this  work  could  average 
$5.64  million  per  aircraft.  Navy  officials  explained  that  they  will  decide  on 
an  aircraft-by-aircraft  basis  whether  to  do  this  work  when  the  aircraft  is 


21While  the  capability  upgrades  may  enhance  mission  effectiveness,  they  are  not 
essential  for  the  aircraft  to  fly  the  additional  1 ,400  hours. 
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inspected  just  prior  to  beginning  service-life  extension.  Navy  officials  told 
us  that  if  an  aircraft  needs  a  lot  of  this  type  of  work,  the  Navy  may  decide 
not  to  extend  the  service  life  of  that  particular  aircraft,  but  rather  substitute 
another  one  in  its  place.  Given  the  uncertainties,  the  costs  associated 
with  this  additional  work  are  not  included  in  the  Navy’s  $2.19  billion 
estimate  nor  are  they  included  in  the  Navy’s  spending  plans  through  fiscal 
year  2017.  The  purpose  of  a  sensitivity  analysis  is  to  identify  the  effects 
on  the  estimate  of  changing  key  factors  and  to  show  a  resulting  range  of 
possible  costs.  Further  analysis  could  show  the  likelihood  of  the  Navy 
incurring  some  or  all  of  the  additional  costs.  Without  an  assessment  of  the 
effect  on  the  total  costs  that  would  be  incurred  if  some  of  the  150  aircraft 
need  additional  work  at  an  additional  cost,  decision  makers  will  not  know 
the  full  range  of  potential  costs. 

Finally,  the  Navy  did  not  compare  the  estimate  to  an  independently 
developed  cost  estimate  such  as  one  that  could  have  been  requested 
from  the  Naval  Center  for  Cost  Analysis.  Navy  officials  stated  that  the 
estimate  was  reviewed  extensively  within  the  Navy  and  that  an 
independent  estimate  was  not  required  because  the  service-life  extension 
is  a  modification  of  an  existing  aircraft  and  is  not  a  separate  acquisition 
program.  Even  if  not  required,  Navy  officials  said  that  they  could  have 
requested  the  Naval  Center  for  Cost  Analysis  to  develop  an  independent 
estimate  for  this  program.  In  our  prior  work,  we  have  found  that  an 
independent  cost  estimate  is  considered  to  be  one  of  the  most  reliable 
validation  methods.  If  the  Navy’s  estimate  were  validated  by  an 
independent  cost  estimate,  decision-makers  could  place  more  credibility 
in  the  estimate. 


Conclusions 


The  Air  Force  and  the  Navy  decided  to  upgrade  and  extend  the  service 
life  of  selected  F-16  and  F/A-18  aircraft  to  provide  a  capability  and 
capacity  “bridge”  until  the  F-35  is  available  in  sufficient  numbers.  The  cost 
to  do  so  is  already  estimated  to  total  almost  $5  billion  and  likely  will 
increase  due  to  slippage  in  the  F-35  program  and  increases  in  program 
scope.  In  many  respects,  the  Air  Force  and  Navy  cost  estimates  we 
evaluated  were  developed  in  accordance  with  some  best  practices. 
Specifically,  we  found  the  estimates  to  be  well-documented  and  accurate 
and  mostly  comprehensive.  However,  the  Air  Force  and  Navy  did  not 
follow  some  important  best  practices  and  as  a  result  did  not  clearly 
identify  the  potential  total  costs  that  may  result  as  the  programs  evolve. 
For  example,  without  an  analysis  that  includes  the  potential  for  increases 
in  the  number  of  aircraft  in  the  program  or  that  clearly  identifies  the  cost 
for  additional  work  that  may  be  required  on  some  of  the  aircraft,  decision 
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makers  are  not  aware  of  how  much  total  costs  could  increase  if  these 
events  occur.  Further,  unless  the  services  reconcile  the  program  cost 
estimate  with  an  independent  cost  estimate,  there  is  an  increased  risk 
that  the  actual  costs  will  exceed  the  estimate.  According  to  the  GAO  Cost 
Estimating  and  Assessment  Guide,  independent  estimates  are  usually 
higher  and  more  accurate  than  program  estimates  and  therefore  if  a 
program  estimate  is  close  to  an  independently  developed  estimate,  one 
can  be  more  confident  that  the  estimate  is  credible.  The  guide  further 
explains  that  there  are  various  types  of  independent  cost  estimate 
reviews  that  can  be  performed,  including  seeking  an  independent 
estimate.  Since  these  are  multibillion-dollar  investments  that  are  critical  to 
maintaining  fighter  capacity,  an  independent  review  of  service  estimates 
or  an  independently-developed  estimate — which  could  be  performed  by 
the  respective  service  cost-estimating  agencies — would  provide 
assurance  about  the  likely  full  costs.  Without  fully  credible  cost  estimates, 
service  and  congressional  decision-makers  will  not  have  reasonable 
confidence  of  knowing  the  potential  range  of  costs  to  upgrade  and  extend 
the  service  life  of  selected  F-16  and  F/A-18  aircraft  as  they  make 
resource  and  trade-off  decisions,  develop  and  review  budget  requests, 
and  assess  the  programs’  affordability. 


Recommendations 
Executive  Action 


Direct  the  Secretary  of  the  Air  Force  to  update  its  cost  estimates,  and  in 
doing  so: 

•  include  in  its  sensitivity  analysis  an  assessment  of  the  range  of 
possible  costs  if  the  capability-upgrade  and  service-life  extension 
programs  are  expanded  to  more  than  300  aircraft  including  up  to  the 
maximum  of  650  aircraft,  and 

•  obtain  an  independent  review  of  the  updated  cost  estimates. 

Direct  the  Secretary  of  the  Navy  to  update  its  cost  estimates  and  in  doing 
so: 


for  1"°  'mProve  future  updates  of  Air  Force  and  Navy  cost  estimates  for 

upgrading  and  extending  the  service  life  of  selected  F-16s  and  F/A-18s 
and  to  improve  the  ability  of  decision-makers  to  assess  the  potential  total 
costs,  we  recommend  that  the  Secretary  of  Defense  take  the  following 
four  actions: 


•  include  in  its  sensitivity  analysis  an  assessment  of  the  range  of 
possible  costs  for  extending  the  service  life  of  other  nonstructural 
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components  that  are  becoming  obsolete  and  capability  upgrades  that 
may  be  required  for  some  of  the  150  aircraft,  and 
•  obtain  an  independent  review  of  the  updated  cost  estimates. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  written 
comments  on  the  draft  of  this  report,  DOD  agreed  with  all  four 
recommendations.  DOD’s  comments  are  reprinted  in  their  entirety  in 
appendix  III.  DOD  also  provided  technical  comments,  which  we 
incorporated  into  the  report  as  appropriate. 

Regarding  the  first  recommendation  for  improving  future  updates  of  the 
Air  Force  cost  estimates,  DOD  concurred,  stating  that  the  Air  Force  had 
updated  planned  force  structure  requirements  to  upgrade  and  extend  the 
service  life  of  50  aircraft  in  addition  to  the  original  300  aircraft  and 
therefore  estimated  costs  for  350  aircraft.  Also,  the  department  stated 
that  while  these  programs  could  be  expanded  beyond  350  aircraft,  further 
cost  excursions  are  premature  until  F-35  production  has  matured.  The 
department  stated  that  it  would  update  its  cost  estimates  with  another 
sensitivity  analysis  containing  an  assessment  of  the  range  of  possible 
costs  if  force  structure  requirements  change  further.  While  we 
acknowledge  that  estimating  costs  for  350  aircraft  is  a  step  in  the  right 
direction,  an  assessment  of  how  much  the  total  cost  may  increase  up  to 
the  maximum  of  650  aircraft  is  needed  to  provide  transparency  on  the  full 
range  of  costs,  which  could  be  significant — potentially  totaling  up  to  $5.53 
billion.  DOD  also  noted  that  further  cost  excursions  are  premature  until  F- 
16  structural  testing  is  complete.  However,  Air  Force  officials  explained 
during  our  review  that  data  from  testing  are  available  throughout  the 
testing  period  and  that  they  could  use  this  information  to  update  cost 
estimates  before  testing  is  completed  in  2016.  Until  the  Air  Force 
completes  analyses  to  show  the  potential  range  of  costs,  decision  makers 
will  continue  to  lack  crucial  data  on  how  much  total  costs  could  increase 
as  they  make  resource  and  trade-off  decisions. 

Regarding  the  second  recommendation  for  improving  future  updates  of 
the  Air  Force  cost  estimates  by  obtaining  an  independent  review  of  the 
updated  cost  estimates,  DOD  agreed,  stating  that  the  Air  Force  Cost 
Analysis  Agency  had  reviewed  the  estimates  for  the  Air  Force’s  fiscal 
year  2014  budget  development  process  and  that,  if  further  cost 
assessments  are  warranted,  the  Air  Force  would  submit  the  estimates  for 
an  independent  review.  An  independent  review  of  cost  estimates  is 
important  because  a  program  that  has  not  been  reconciled  with  an 
independent  estimate  has  an  increased  risk  of  proceeding  underfunded 
since  an  independent  estimate  provides  an  objective  and  unbiased 
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assessment  of  whether  the  program  estimate  can  be  achieved.  As  the 
Air  Force  updates  its  cost  estimates  for  these  $2.61  billion  programs,  up- 
to-date,  independently  validated  cost  estimates  would  provide  decision¬ 
makers  assurance  that  the  programs’  estimated  cost  can  be  relied  upon 
as  they  assess  program  affordability. 

Regarding  the  two  recommendations  for  improving  future  updates  of  the 
Navy  cost  estimates,  DOD  concurred  but  did  not  explain  when  or  what 
specific  actions  it  intends  to  take  to  implement  these  recommendations. 
We  are  encouraged  that  the  department  agrees  these  are  important,  valid 
steps  to  take  and  further  believe  that  it  is  important  for  the  Navy  to 
implement  these  recommendations  in  a  timely  manner  to  facilitate  review 
and  implementation  of  these  programs  since  the  programs  are  estimated 
to  cost  $2.19  billion  and  could  increase  an  average  of  $5.64  million  per 
aircraft. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  Defense,  the  Secretary  of  the  Air  Force,  and 
the  Secretary  of  the  Navy.  The  report  also  is  available  at  no  charge  on  the 
GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (404)  679-1816  or  pendletonj@gao.gov.  Contact  points  for 
our  Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found 
on  the  last  page  of  this  report.  Staff  members  making  key  contributions  to 
this  report  are  listed  in  appendix  IV. 


John  H.  Pendleton 
Director 

Defense  Capabilities  and  Management 
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Appendix  I:  Scope  and  Methodology 


To  describe  the  Air  Force’s  and  the  Navy’s  plans  to  upgrade  and  extend 
the  service  life  of  selected  F-16s  and  F/A-18s,  we  reviewed  the  services’ 
documentation  of  the  programs’  purpose  and  scope  including  the 
underlying  analysis  of  alternative  approaches.  To  identify  the  Air  Force’s 
and  the  Navy’s  estimated  costs  to  upgrade  and  extend  the  service  life  of 
F-16s  and  F/A-18s,  we  obtained  the  services’  cost  estimates  for  these 
programs  and  obtained  any  updates  to  these  estimates.  The  Air  Force 
updated  its  initial  estimate  to  support  development  of  the  fiscal  year  2013 
budget.  The  Navy’s  cost  estimate  was  reported  to  Congress  in  May  2011. 
All  of  the  cost  estimates  from  these  sources  were  converted  to  fiscal  year 
2013  dollars  using  the  indexes  published  by  the  Office  of  the  Secretary  of 
Defense  (Comptroller)  in  the  National  Defense  Budget  Estimates  for 
Fiscal  Year  2013,  commonly  referred  to  as  the  “Green  Book”.1 

To  assess  the  extent  to  which  the  cost  estimates  exhibited  characteristics 
of  a  high-quality  cost  estimate,  we  compared  how  the  estimates  were 
developed  to  the  cost  estimating  best  practices  outlined  in  the  GAO  Cost 
Estimating  and  Assessment  Guide.2  The  guide  is  a  compilation  of  best 
practices  that  federal  cost-estimating  organizations  and  industry  use  to 
develop  and  maintain  reliable  cost  estimates  that  management  can  use 
for  making  informed  decisions.  The  guide  describes  19  best  practices  for 
developing  a  comprehensive,  well-documented,  accurate,  and  credible 
cost  estimate — the  four  characteristics  of  a  high-quality  cost  estimate.  We 
analyzed  the  extent  to  which  the  Air  Force  and  Navy  cost  estimates  met 
each  of  these  four  characteristics  and  assigned  each  a  rating  of  not  met, 
minimally  met,  partially  met,  substantially  met,  or  met.3  The  overall  rating 


Constant  dollars  measure  the  value  of  purchased  goods  and  services  at  price  levels  that 
are  the  same  as  those  in  the  base  or  reference  year.  Constant  dollars  do  not  contain  any 
adjustments  for  inflationary  changes  that  have  occurred  or  are  forecast  to  occur  outside 
the  base  year.  Therefore,  the  changes  in  these  dollar  amounts  will  not  be  due  to  inflation. 

2GAO,  GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for  Developing  and 
Managing  Capital  Program  Costs,  GAO-09-3SP  (Washington,  D.C:  March  2009).  We 
performed  this  comparison  for  the  Air  Force  estimate  for  capability  upgrades,  the  Air  Force 
estimate  for  service-life  extension,  and  the  Navy  estimate  for  service-life  extension.  We 
did  not  perform  this  comparison  for  the  Navy  estimates  of  average  cost  per  aircraft  for 
obsolescence  work  or  capability  upgrades. 

3Not  met  means  the  military  service  provided  no  evidence  that  satisfies  any  of  the 
criterion;  minimally  met  means  the  military  service  provided  evidence  that  satisfies  a  small 
portion  of  the  criterion;  partially  met  means  the  military  service  provided  evidence  that 
satisfies  about  half  of  the  criterion;  substantially  met  means  the  military  service  provided 
evidence  that  satisfies  a  large  portion  of  the  criterion;  and  met  means  the  military  service 
provided  complete  evidence  that  satisfies  the  entire  criterion. 
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Appendix  I:  Scope  and  Methodology 


for  each  characteristic  was  based  on  the  average  rating  of  the  best 
practices  for  each  characteristic.  We  also  held  detailed  discussions  with 
service  officials  and  reviewed  program  documentation  to  identify  key 
factors  that  could  affect  the  potential  total  costs  such  as  changes  in 
aircraft  quantities  or  additional  work  to  maintain  aircraft  mission 
effectiveness  that  may  not  have  been  included  in  the  estimates.  We 
shared  with  program  officials  our  cost  guide,  the  criteria  against  which  we 
evaluated  the  program’s  estimated  cost,  as  well  as  our  preliminary 
findings.  When  warranted,  we  updated  our  analyses  on  the  basis  of  the 
agency  response  and  additional  documentation  that  was  provided  to  us. 
Finally,  we  corroborated  our  analyses  in  interviews  with  service  officials 
responsible  for  developing  the  cost  estimates. 

We  conducted  our  work  at  the  Office  of  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology  and  Logistics;  Office  of  the  Chief  of  Staff  of 
the  Air  Force,  Directorate  of  Force  Application;  Ogden  Air  Logistics 
Center  at  Hill  Air  Force  Base;  F-16  System  Program  Office  at  Wright- 
Patterson  Air  Force  Base;  Air  Combat  Command;  Air  Force  Cost  Analysis 
Agency;  Air  Force  Fleet  Viability  Board;  Office  of  the  Deputy  Assistant 
Secretary  of  the  Navy  for  Air  Programs;  Headquarters  United  States 
Marine  Corps;  Naval  Air  Systems  Command;  and  the  Naval  Center  for 
Cost  Analysis. 

We  conducted  this  performance  audit  from  May  2011  to  November  201 2 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings,  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Appendix  II:  Air  Force  Cost  Estimates  for 
Upgrading  and  Extending  the  Service  Life  of 
Selected  F-16  Aircraft  in  Then- Year  Dollars 


Dollars  in  millions 

Fiscal  year 

2011 

2012 

2013 

2014 

2015 

2016 

2017 

2018 

2019 

2020 

2021 

2022 

Total 

Service-life 

Extension 

$18.80 

$0.80 

$  8.86 

$12.10 

$14.78 

$50.59 

$110.88 

$167.80 

$220.50 

$188.10 

$104.30 

$0.00 

$897.51 

Capability 

upgrades 

0.00 

12.00 

69.70 

129.50 

98.20 

26.50 

201.85 

355.90 

379.70 

381.30 

269.00 

41.20 

$1,964.85 

Total 

$18.80 

$12.80 

$78.56 

$141.60 

$112.98 

$77.09 

$312.73 

$523.70 

$600.20 

$569.40 

$373.30 

$41.20 

$2,862.36 

Source:  Air  Force  data. 


Notes:  The  numbers  in  this  table  are  in  then-year  dollars.  The  numbers  in  the  report  text  are  in  fiscal 
year  2013  dollars. 

The  Navy  estimate  was  reported  as  a  cost  per  aircraft  in  fiscal  year  201 1  dollars.  Therefore,  there  is 
not  comparable  data  for  showing  the  Navy  estimate  in  then-year  dollars. 
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Appendix  III:  Comments  from  the 
Department  of  Defense 


ACQUISITION 


ASSISTANT  SECRETARY  OF  DEFENSE 


3015  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3015 


Mr.  John  H.  Pendleton  110712 

Director,  Defense  Capabilities  and  Management 

U.S.  Government  Accountability  Office 

441  G  Street  NW 

Washington,  DC  20548 

Dear  Mr.  Pendleton: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability 
Office  (GAO)  Draff  Report,  GAO-13-51,  "FIGHTER  AIRCRAFT:  Better  Cost  Estimates 
Needed  for  Extending  the  Service  Life  of  Selected  F-16s  and  F/A-l  8s,"  dated  September  28, 
2012,  (GAO  Code  351600).  The  Department  appreciates  the  effort  of  the  GAO  and  the 
opportunity  to  comment  on  the  draff  report. 

The  Department  concurs  with  the  four  recommendations  contained  in  the  draff  report. 
Detailed  comments  on  the  report  recommendations  are  enclosed  and  the  Department  looks 
forward  to  continuing  to  work  with  the  GAO  as  we  continue  to  improve  our  acquisition 
processes. 

Sincerely, 


Enclosure: 
As  stated 
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of  Defense 


GAO  DRAFT  REPORT  DATED  SEPTEMBER  28, 2012 
GAO-13-51  (GAO  CODE  351600) 

“FIGHTER  AIRCRAFT:  BETTER  COST  ESTIMATES  NEEDED 
FOR  EXTENDING  THE  SERVICE  EIFF,  OF  SELECTED  F-16S  AND 
F/A-18S” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  To  improve  future  updates  of  Air  Force  cost  estimates  for 
upgrading  and  extending  the  service  life  of  selected  F-16s  and  to  improve  the  ability  of  decision¬ 
makers  to  assess  the  potential  total  costs,  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Air  Force  to  update  its  cost  estimates  and  include  in  its  sensitivity  analysis 
an  assessment  of  the  range  of  possible  costs  if  the  capability  upgrade  and  service  life  extension 
programs  are  expanded  to  more  than  300  aircraft  including  up  to  the  maximum  of  650  aircraft. 

DoD  RESPONSE:  Concur.  The  Air  Force  Cost  Analysis  Agency  (AFCAA)  conducted  a 
thorough  Non-Advocacy  Cost  Assessment  (NACA)  in  2012  for  the  F-16s'  Service-Life 
Extension  Program  (SI.F.P)  and  Combat  Avionics  Programmed  Extension  Suite  (CAPES) 
upgrade.  Planned  force  structure  requirements,  originally  met  with  300  modernized  aircraft  and 
recently  updated  to  350  aircraft,  guided  the  assessment.  While  the  Air  Force  has  stated  it  has  the 
ability  and  capacity  to  scale  up  fighter  modernization  programs  further,  at  this  point  further  cost 
excursions  are  premature  until  F-16  structural  testing  is  complete  and  F-35  production  has 
matured.  If  force  structure  requirements  change  further,  the  Air  Force  will  update  its  cost 
estimates  with  another  sensitivity  analysis  containing  an  assessment  of  the  range  of  possible 
costs  in  the  next  annual  programming  and  budgeting  cycle  for  which  such  an  update  is 
appropriate. 

RECOMMENDATION  2:  To  improve  future  updates  of  Air  Force  cost  estimates  for 
upgrading  and  extending  the  service  life  of  selected  F-16s  and  to  improve  the  ability  of  decision¬ 
makers  to  assess  the  potential  total  costs,  GAO  recommends  that  the  Secretary  of  Defease  direct 
the  Secretary'  of  the  Air  Force  to  update  its  cost  estimates  and  obtain  an  independent  review  of 
the  updated  cost  estimates. 

DoD  RESPONSE:  Concur.  SLEP  and  CAPES  qualify  as  Acquisition  Category  (ACAT)  III  and 
ACAT  II  respectively.  ACAT  II  and  III  programs  are  executed  by  the  Sendee  Component 
Acquisition  Executive,  and  therefore  by  charter  AFCAA  is  the  responsible  cost  analysis  agency. 
AFCAA  has  the  capability  to  perform  independent  analysis  via  NACAs  on  selected  programs.  A 
rigorous  process  in  coordination  with  the  Air  Force  Corporate  Structure  determines  which 
programs  require  such  an  assessment  to  support  Air  Force  decisions.  The  F-16  SLEP/CAPES 
programs  were  selected  for  such  a  review  for  the  Air  Force's  14  POM  submission  and  the  results 
were  presented  to  the  GAO  during  the  course  of  their  review.  If  further  cost  assessments  are 
warranted,  the  Air  Force  will  submit  the  estimates  to  an  independent  review. 
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Appendix  III:  Comments  from  the  Department 
of  Defense 


RECOMMENDATION  3:  To  improve  future  updates  of  Navy  cost  estimates  for  upgrading 
and  extending  the  sendee  life  of  selected  F/A- 1  Xs  and  to  improve  the  ability  of  decision-makers 
to  assess  the  potential  total  costs,  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Secretary7  of  the  Navy  to  update  its  cost  estimates  and  include  in  its  sensitivity7  analysis  an 
assessment  of  the  range  of  possible  costs  for  extending  the  service  life  of  other  non-structural 
components  that  are  becoming  obsolete  and  capability  upgrades  that  may  be  required  for  some  of 
the  1 50  aircraft. 

DoD  RESPONSE:  Concur 

RECOMMENDATION  4:  To  improve  future  updates  of  Navy7  cost  estimates  for  upgrading 
and  extending  the  sendee  life  of  selected  F/A-18s  and  to  improve  the  ability  of  decision-makers 
to  assess  the  potential  total  costs.  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Secretary7  of  the  Navy  to  update  its  cost  estimates  and  obtain  an  independent  review  of  the 
updated  cost  estimates. 

DoD  RESPONSE:  Concur. 
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Appendix  IV:  GAO  Contact  and  Staff 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

Please  Print  on  Recycled  Paper. 

